In a financial institution, a banknote handling device is used at the time of transaction involving payment and receipt of money between a customer and a staff (a so-called teller) of the financial institution (for example, JP-A-2004-145600). Here, in case of deposit transaction at the financial institution, it is generally treated that the customer side has the ownership of banknotes until the transaction is established. Therefore, the banknote handling device adopts a structure that even after the reception of the banknotes from the customer in the deposit transaction, the banknotes are stored in a temporary holding unit to keep them returnable until the transaction is established, and after the transaction is established, the banknotes are stored in the storage compartments by denomination. The banknote handling device also has a function to exclude as a reject banknote a banknote having a stain, a shape or the like not satisfying a prescribed standard and not suitable for transaction, from the transaction object in order to perform the banknote transaction smoothly.